Embodiments described herein generally relate to three-dimensional (3D) image data sets and in particular how to edit curves that follow a path relative to an anatomical feature of interest in 3D image data sets.
In the medical field, three-dimensional (3D) image data sets, i.e. volume data sets, are collected by a variety of techniques—referred to as modalities in the field—including computer assisted tomography (CT), magnetic resonance (MR), single photon emission computed tomography (SPECT), ultrasound and positron emission tomography (PET).
When displaying an image, particular signal values associated with an image data set can be associated with particular display brightness and also, in the case of non-grayscale images, colors to assist visualization. This association or mapping is performed when using data from a 3D data set (voxel data set) to compute a 2D data set (pixel data set) which represents a 2D projection of the 3D data set for display on a computer screen or other conventional 2D display apparatus. This process is known as volume rendering or more generally rendering.
A Widely Used Rendering Technique is Multi Planar Reformatting (MPR).
MPR is a way of presenting planar cross-sectional views through volume data to allow viewing of the data in any planar orientation independent of the orientation of the underlying data, which may have been collected in slices in the case of CT or MR.
However, in medical imaging many anatomical features of interest are non-planar. Consequently, a planar imaging method such as MPR is not always optimal, since a planar view may not allow visibility of a whole non-planar feature. For example, a surgeon may wish to study an artery along its entire length. The artery will follow a complex path in three-dimensions, so that any slice through the volume around the artery will only show part of it. The same problem arises for other curved anatomical structures, such as bronchi, the colon, the spine, and dental structures.
The technique of curved MPR was developed to address this limitation of conventional (planar) MPR.
Curved MPR allows curved planes to be defined within the data set and then viewed as a two-dimensional image. This allows, for example, curved, non-planar anatomical structures such as blood vessels or the colon, which cannot be fully seen in a single planar slice through the data set, to be shown in a single view. To generate the curved view, a three-dimensional curve which follows the structure of interest is defined through the volume data set representing the patient's body. The curved MPR view is then generated by extruding the curve in a particular direction to produce a curtain-like sheet in the three-dimensional volume, and then flattening the sheet and presenting it as a two-dimensional image.
It is noted that in the literature curved MPR is sometimes referred to as CPR (Curved Planar Reformatting).
Further views, known as cross-curve MPR views can be presented once the curve has been defined; these views are planar cross-sections through the data set which are perpendicular to the curve at a selected point.
By way of definition and to avoid confusion, any subsequent reference to “MPR” in this document which is not prefaced with “curved” or “cross-curve” is a reference to conventional, planar MPR.
Moreover, we use saggital, coronal and transverse according to their conventional medical definitions, for example when referring to particular views. Moreover, we use the term oblique to refer to a view or direction or plane which is not precisely saggital, coronal or transverse. Reference to an oblique MPR view is thus reference to an MPR view at an arbitrary viewing angle and in an arbitrary view plane. Reference to an oblique transverse view means a view that is substantially, but not exactly, transverse.
In the case of a vessel, intestine or other generally tubular anatomical feature, the curve relevant for the curved MPR view is a centerline. For other structures, such as to follow the curvature of the spine or to follow the line of teeth around a jaw, the term centerline is perhaps a less appropriate label for the curve.
By way of definition we use the term lumen as a generic label for a tubular organ, such as a blood vessel, artery or vein, or an intestine or the colon. To find the centerline of a lumen it is standard practice to use an automatic centerline finding algorithm. Since a curved MPR view is based on the centerline, any errors in the centerline relative to the underlying anatomical feature of interest are likely to cause the curved MPR image to appear differently to how the viewer imagines it should.
FIG. 1 illustrates one common type of error produced by an automated centerline finding algorithm for an artery. Bends in the artery cause the automatic centerline finding algorithm used here to cut the corner. The centerline algorithm is said to have followed a “racing line” around the bend—which means moving away from the center of the artery and approaching the apex of the bend. With a racing line around a bend, and assuming the artery has a perfectly circular diameter which is constant along its length, the curved MPR view will show that the artery diameter reduces (or enlarges) where there are bends, but the position of the bends and how tight they are will not necessarily be clear from the curved MPR view. Consequently in the curved MPR view, the diameter variations in the artery may not appear to be simply correlated with bends. To the clinician, the artery therefore may appear to have a stenosis (or aneurysm) when it is in fact perfectly healthy.
FIGS. 2 and 3 illustrate another common type of error produced by an automated centerline finding algorithm for an artery when areas of the artery wall have significant plaque deposits. Each of FIG. 2 and FIG. 3 is a collection of views (transverse MPR, coronal MPR, cross-curve MPR and curved MPR view) of an artery with the centerline computed by an automatic centerline finding algorithm marked and shown as a dark line, and a portion of the artery with a plaque-induced error circled.
Some automatic centerline finding algorithms will tend to confuse the plaque with the artery's tubular structure, and thus the centerline will jump from the center of the artery where there is no plaque, to the center of the plaque where there is significant plaque on one side of the artery. Moreover, since the plaque may be at any angular position on the artery wall as viewed in cross-section, the centerline can randomly move around the wall in a kind of zig-zag spiral. The plaque-induced centerline error is shown in an extreme form in the cross-curve MPR views of FIGS. 2 and 3, where the plaque is white and the artery is gray. It is evident that the centerline is in the middle of the plaque and on the very edge of the artery. In the coronal MPR views of FIGS. 2 and 3, the zig-zag effect of following the plaque is visible and it is quite clear that the centerline is misplaced. On the other hand, in the transverse MPR views of FIGS. 2 and 3, the centerline gives the false impression of being roughly correct.
The complex and non-intuitive nature of the transform involved with curved MPR and its relation to any errors in the centerline make this a really difficult problem from the user perspective. For example, a bend might be visible in the curved MPR view which is somewhat similar to the real bend, but not the same. This is because a curved MPR view does not simply straighten out a curve, the curved MPR computation being more complex than that. A user tends to think of a curved MPR view as a (planar) MPR view which has a viewing direction parallel to the viewing direction from which the standard MPR plane (coronal, saggital or transverse) is displayed, since this helps the user to put the curved MPR image into context within the patient. However, this way of thinking is a false friend when it comes to the user comparing the curved MPR view with the actual underlying data. In the general case, it is simply beyond a normal user's intuition to follow the transform that has been carried out to compute the curved MPR view and at the same time think about how different kinds of common errors in the centerline positioning will distort the curved MPR view.
Because of these known problems, it is necessary for the centerline computed by the automated centerline finding algorithm to be reviewed by an expert and if necessary manually corrected by intervention via the graphical user interface (GUI). The GUI is provided with standard CAD (computer aided design) tools to aid the editing. An editing session will include identification of a portion of the centerline that needs to be corrected and provision of a graphical user interface tool, such as a “nudge” tool, to allow the centerline to be moved.
Reviewing and editing centerlines computed by an automatic centerline finding algorithm is quite time consuming with the editing tools provided in existing rendering applications. Moreover, the existing editing tools can be confusing. Because of these factors, the review and editing of centerlines computed by automatic centerline finding algorithms is typically delegated by the clinician to a technician.
Some existing editing solutions are based on editing the centerline in the cross-curve MPR or curved MPR views. For example, the rendering application “syngo” (registered trade mark) marketed by Siemens Medical Systems, Inc. offered such a feature in version “CT 2005A”.
FIG. 4 shows the layout of views used in several subsequent figures, which is one potential layout for a rendering/visualisation application. Transverse, coronal and saggital MPR views are shown in a strip from left to right at the top of the display. A curved MPR view is shown larger than the other views on the right lower part of the display. On the left middle part of the display, a cross-curve MPR view is shown, and on the left lower part of the display a volumetric view is shown which is a perspective view with significant saggital, coronal and axial components. The smallest views are three views labeled P.H. and these are placeholders for stenosis measurement views associated with this particular example layout.
An example editing session based on editing the cross-curve MPR view is now described.
FIGS. 5A, 5B and 5C are screen shots in the format of FIG. 4. FIGS. 5A, 5B and 5C illustrate an example centerline editing session based on editing the cross-curve MPR views. FIG. 5A shows the starting state before the editing session is commenced, FIG. 5B shows the layout after a first edit. FIG. 5C shows the layout after a second edit.
In FIG. 5A, the centerline has passed through some calcified plaque and needs to be moved to its correct position following the artery's true centerline. To make the correction, the user initiates an editing session and, as shown in FIG. 5B, drags the centerline to the center of the vessel lumen with the aid of the cross-curve MPR view. However, as can be seen in the coronal MPR view of FIG. 5B, this manipulation has caused a sharp spike in the centerline, i.e. a new error. Moreover, this spike has caused artificial elongation of the curved MPR view. To correct the spike, as shown in FIG. 5C, the user changes the cross-curve MPR view from oblique saggital to oblique transverse and flattens out the spike by dragging the centerline in this view. The drag has the desired effect on the spike, but further artificially elongates the curved MPR view.
The basic problem which this example shows is that, since cross-curve MPR views (and also curved MPR views) are themselves by-products of the existing centerline, editing these views is inherently likely to cause confusion to the user. It is not uncommon that an attempt to correct a portion of the centerline reaches such a confused state that the user has to restart the editing session.
FIGS. 6A and 6B are screen shots of the top row of views in the layout of FIG. 4 which illustrate an example centerline editing session based on editing the coronal MPR view. FIG. 6A shows the standard planar MPR views (transverse, coronal, sagittal) before an edit. FIG. 6B shows the same views after the edit.
In FIG. 6A, the user sees that the centerline seems roughly (although nor perfectly) correct in the transverse MPR view, but shows significant plaque error in the coronal MPR and sagittal MPR views, where plaque appears bright white and the artery gray. For example, in the coronal view the computed centerline can be seen as being below the true centerline of the artery. In the transverse view the computed centerline can be seen as being more jagged than the true centerline of the artery. The user therefore edits the centerline in the coronal MPR view to follow the artery, not the plaque, as shown in the coronal MPR view of FIG. 6B. The coronal MPR view of FIG. 6B shows some improvement in the centerline position compared with FIG. 6A, but still does not look accurate. However, the transverse MPR view of FIG. 6B, clearly shows the edit has not corrected the centerline completely, since in the transverse MPR view the centerline is still zig-zagging (for example as compared to the transverse and saggital views) to follow plaque. In this example, some positive progress appears to have been made, but clearly the user has more work to do, for example by editing the transverse MPR view.
For editing the centerline using these known approaches, the user therefore needs clear visibility of effects of the centerline edits he is making in the corresponding planes. Due to lack of screen space the user may need to modify the display to in effect magnify the region of interest. For example, the user might need to expand the window containing the view of interest by in effect taking screen space from other views, and potentially “covering” the other views entirely with the expanded view of interest. Alternatively, or in addition, the user might zoom in on a smaller part of the view of interest. However, both these approaches can be inconvenient and cumbersome. For example, with the first approach the user will need to cycle through the different views in series to try to assess the overall impact of the edits in multiple planes (because he can no longer see all views at once). With the second approach, the user may be unable to see the entire portion of centerline of interest at one time and so need to make amendments iteratively for different portions of the centerline in the same view plane.